Enterprise Application Integration (EAI) solutions typically facilitate efficient processing solutions by aggregating a plurality of messages into a message bundle referred to herein as an interchange. For example, the EAI system may include business software that collects data over the course of a day and sends the collected messages to a business partner in an serial data interchange of individual messages once per day. The interchange may be represented, for example, by a flat file or XML, and the interchange passes through a receive pipeline at a receive location where the interchange is processed by an endpoint manager to determine whether each message in the flat file or XML interchange is to be routed to any particular recipients. If a routing is requested, the message(s) are routed to the recipient(s) for further processing.
Exchanging of such data interchanges containing multiple elements is a very common practice in EAI solutions. A simple example of such an interchange is an order placed by a customer on a retailer web site. A typical order consists of more than just a single line item. Also, each line item may represent a document that needs to be handled by a different inventory system. The recipient of the interchange that represents the order may need to split up the interchange into smaller independent messages that correspond to each line item. Those messages or documents then will be sent to corresponding inventory systems.
Very often the interchanges contain errors that cause the process of splitting the interchange into individual messages to fail. The range of errors may be very wide: from format mistakes or inconsistencies to business logic problems. The recipient of the interchange needs to be able to determine the erroneous elements and to filter them out from the interchange so that the rest of the elements are still processed successfully. Unfortunately, given the format of the interchanges (typically XML or flat file) and potentially their large sizes, it becomes very difficult and in some cases not possible to recover from a failure in the middle of the processing and to continue processing from the next good element.
Conventional EAI products, such as Microsoft's BizTalk Server family, IBM WebSphere, BEA WebLogic, Tibco, and the like typically handle such errors in one of two ways:                1. When the first error is encountered during interchange processing, the processing is stopped, all the messages that were already produced are rolled back, the original interchange is suspended, and the operator is informed of the failure, and/or        2. When the first error is encountered during interchange processing, the processing is stopped, the messages that were already produced are allowed to continue processing, the remainder of the original interchange is suspended, and the operator is informed of the failure.        
For example, FIG. 1 is a simplified illustration of an EAI solution where a flat file or XML interchange 10 containing a plurality of serialized messages or documents DOC1, DOC 2, DOC 3, and DOC4 is to be processed by an endpoint manager 20 including a receive pipeline 30 at a receive location 40. In the example of FIG. 1, it is assumed that the message DOC3 contains a format or some other error that will prevent its successful processing by the endpoint manager 20. In the conventional EAI systems, the first two messages, DOC1 and DOC2, are processed successfully. However, when the processing of DOC3 fails, the processing of the entire flat file or XML interchange 10 is stopped, and either DOC1 and DOC2 are rolled back or DOC1 and DOC2 are allowed to continue processing. In either case, the remainder of the processing of the flat file or XML interchange 10 is suspended and the operator informed of the failure. In the first prior art embodiment, shown in FIG. 2A, nothing is stored in the message queue 50 while the entire flat file or XML file interchange 10 is stored in the suspend queue 60 for recovery processing. On the other hand, in the second prior art embodiment, shown in FIG. 2B, DOC1 and DOC2 are stored successfully in the message queue 50, while processing of DOC3 and DOC4 is not completed. As a result, DOC3 and DOC4 are stored in the suspend queue 60.
Such processing is relatively inefficient because there may be relatively few problematic entries that hold up the processing of a potentially very large data interchange. For example, if DOC3 is followed by hundreds of additional messages in the interchange, little or no processing would be permitted to proceed until the recovery processing were completed. Applicant is not aware of an EAI product that instead enables the interchange processing to proceed, despite problematic messages in the interchange, by suspending only the problematic element in its original form when an error is encountered during interchange processing, searching for the next valid element in the interchange, and resuming processing with the next valid element while, at the same time, the operator is informed of any failures. As illustrated in FIG. 2C, if such a methodology were available and applied to the above example, DOC1, DOC2, and DOC4 would be successfully stored in the message queue 50, while only the message with the error (DOC3) would be sent to the suspend queue 60 for recovery processing.
The processing approach illustrated in FIG. 2C would provide significant processing efficiency, particularly in the case of large interchanges; however, to date, the processing characteristics of typical EAI solutions do not incorporate such an error handling method for processing interchanges. It is desired to provide a technique for graceful handling of the errors encountered during processing of data interchanges with problematic individual elements and for recovering the processing from next good element in the input data stream. The present invention addresses this need in the art.